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ABSTRACT 

Application  of  robotic  technologies  to  aircraft  servicing  is  a  re¬ 
search  initiative  at  the  Air  Force  Institute  of  Technology.  Cur¬ 
rent  efforts  are  concentrated  on  demonstrating  a  concept  for  semi- 
autonomous  ground  based  robotic  refueling.  Initially  we  are  assum¬ 
ing  that  the  aircraft’s  aerial  refueling  port  is  within  the  range  space 
of  the  refueling  manipulator.  Navigation  issues,  while  important,  are 
not  under  investigation.  We  envision  a  shared  system  where  a  hu¬ 
man  positions  the  refueling  arm  in  the  vicinity  of  the  aircraft  and 
then  allows  the  refueling  port  detection  and  nozzle  insertion  to  be 
done  automatically  by  a  combination  of  visual  servoing  and  compli¬ 
ant  control.  A  testbed  has  been  developed  to  support  experimen¬ 
tal  evaluation  of  the  enabling  technologies  necessary  for  autonomous 
refueling.  Initial  evaluations  have  successfully  demonstrated  algo¬ 
rithms  for  port  detection,  visual  servoing,  and  compliant  control  of 
nozzle  motion  and  insertion.  Semi- autonomous  robotic  refueling  is 
now  feasible. 

1  Introduction 

Within  the  Air  Force,  the  design  and  implementation  of  autonomous 
robotic  systems  capable  of  performing  complex  tasks  will  be  a  ma¬ 
jor  initiative  in  the  coming  decade  [1].  Because  the  Air  Force  may 
have  occasion  to  perform  combat  operations  in  environments  con¬ 
taminated  by  nerve  agents  or  similar  hazards  and  with  fewer  skilled 
personnel  available,  robotic  aircraft  turnaround  has  been  studied 
[4,3].  One  complex  turnaround  task  that  has  been  focused  on  over 
the  past  several  years  is  autonomous  ground-based  aircraft  refueling 
Ground-based  refueling  currently  requires  several  military  members 
to  accomplish  the  task,  and  in  a  hostile  environment,  refueling  can 
be  cumbersome  and  difficult  [19].  Automated  or  robotic  assisted 
turnaround  will  allow  operators  or  supervisors  to  remain  in  relative 
safety  while  accomplishing  mission  objectives  efficiently.  This  sce¬ 
nario  has  several  advantages,  such  as  lower  personnel  requirements, 
successful  operations  in  normally  impossible  conditions,  and  less  fa¬ 
tigue  and  danger  for  personnel.  While  the  technology  exists  to  field 
a  crude  teleoperated  refueling  system,  the  enabling  technologies  for 
semi- autonomous  or  fully  autonomous  operation  are  not  mature. 

Initial  research  in  the  area  of  autonomous  ground-based  refu¬ 
eling  was  conducted  by  the  Flight  Dynamics  Laboratory  at  Wright- 
Patterson  AFB.  Their  research  determined  that  the  air  refueling  port 
is  the  optimal  place  to  refuel  aircraft  when  utilizing  a  robotic  refu¬ 
eling  system.  This  decision  was  based  on  accessibility  of  the  port 
and  minimal  procedural  requirements  (i.e.,  no  removal  of  fuel  caps, 
ease  of  connecting  fuel  hoses)  for  refueling  at  this  location  [19].  Fig¬ 
ure  1  prQvides  an  illustration  of  this  proposed  robotic  ground-based 
refueling  scheme.  A  fuel  truck  is  driven  into  the  close  vicinity  of 
the  aircraft  and  a  robotic  refueling  boom  is  placed  over  the  aircraft. 
The  operation  of  the  robotic  refueling  manipulator  in  the  vicinity  of 


the  target  aircraft  could  be  automated  or  left  under  manual  control 
to  locate  the  refueling  port,  position,  and  then  insert  the  refueling 
nozzle  into  the  port. 

A  robotics  research  initiative  at  the  Air  Force  Institute  of  Tech¬ 
nology  ( AFIT)  is  the  evaluation  of  the  enabling  technologies  required 
for  autonomous  operation  of  the  port  location  and  nozzle  insertion 
portions  of  the  robotic  refueling  task.  Our  approach  advocates  an  in¬ 
tuitive  integration  of  both  vision  and  control  technologies.  The  basic 
premise  is  to  let  each  technology  do  what  it  does  well  Vision  sys¬ 
tems  are  good  at  providing  coarse  position  information.  Compliant 
control  schemes  utilize  force  information  to  compensate  for  errors  in 
goal  position.  Proper  combination  will  produce  a  simple  autonomous 
system  for  mating  the  refueling  arm  and  aircraft  port. 

In  our  scenario,  visual  information  regarding  identification  and  lo¬ 
cation  of  the  refueling  port  is  fed  back  to  the  robot  controller  which 
develops  trajectory  information  needed  for  robot  motion.  The  au¬ 
tonomous  refueling  boom  will  use  visual  information  to  close  the 
loop  around  the  motion  control  problem  forming  a  visual  servoing 
system.  The  role  of  the  visual  servoing  system  is  to  place  the  manip¬ 
ulator  (boom)  in  contact,  or  close  proximity,  with  the  refueling  port 
slipway  [20].  An  impedance  control  scheme  [4,18]  provides  the  neces¬ 
sary  compliance  for  smooth  motion  along  the  refueling  port  slipway 
and  finally,  insertion.  Exact  knowledge  of  port  location  is  not  re¬ 
quired  The  inevitable  misalignment  between  robotic  refueler  and 
port,  caused  by  errors  in  calibration  and  kinematic  transformations, 
is  inherently  accounted  for.  The  issue  of  misalignment  compensation 
cannot  be  overlooked.  A  conventional  robot  controller  is  designed  to 
eliminate  all  position  error  by  increasing  the  output  torque  Increas¬ 
ing  the  output  torque  when  the  robot  is  jammed  leads  to  instability 
and  potential  damage  to  the  contact  surface  and/or  the  manipula¬ 
tor  drive  system  For  a  robotic  refueling  application  the  slipway  is 
already  hardened  so  the  primary  concern  is  additional  wear  of  the 
manipulator  leading  to  decreased  reliability. 

The  goal  of  the  refueling  research  project  during  the  last  year  was 
to  demonstrate  the  component  technologies  for  our  robotic  refueling 
concept.  The  project  was  divided  into  two  areas;  visual  servoing  and 
compliant  control.  The  remainder  of  this  paper  describes  the  research 
accomplishments  that  lead  to  a  successful  concept  demonstration. 
Section  two  describes  the  refueling  testbed.  Visual  servoing  research 
is  the  subject  of  section  three  while  implementation  and  analysis  of 
compliant  control  of  the  nozzle  is  presented  in  section  four.  Future 
research  directions  are  discussed  in  section  five.  Conclusions  are 
presented  in  section  six.  A  more  detailed  description  of  our  results 
are  in  [18,20]. 

2  Refueling  Testbed 

Experimental  evaluation  of  theoretical  concepts  is  a  vital  component 
of  robotics  research.  The  Air  Force  Institute  of  Technology  Robotic 
Systems  Laboratory  has  the  facilities  to  demonstrate  and  evaluate 
robotic  technology  for  Air  Force  applications.  A  block  diagram  of 
the  laboratory  systems  used  to  support  robotic  refueling  research  is 
shown  in  Figure  2. 
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2.1  Manipulator 

The  key  component  of  the  refueling  mock-up  is  the  PUMA-560  robotic 
manipulator  that  simulates  the  refueling  arm.  The  PUMA-560  is  a 
vertically  articulated  six  degree  of  freedom  (DOF)  industrial  robot. 
The  six  degrees  of  freedom  are  three  heavy  links  in  a  serial  configura¬ 
tion  and  three  lighter  links  in  a  roll- bend- roll  wrist.  While  the  PUMA 
design  is  not  suggested  for  eventual  implementations  it  does  provide 
a  rigorous  test  case  for  the  required  technologies.  Our  PUMA  has  a 
highly  modified  control  system  called  ARCADE  [16] .  The  LSI-11/73 
processor  in  the  PUMA  chassis  now  serves  as  a  data  concentrator, 
sending  joint  angles  and  receiving  joint  torques,  for  programs  execut¬ 
ing  on  a  VAXstation  III  (ROBBIE).  ARCADE  supports  the  exper¬ 
imental  evaluation  of  advanced  control  algorithms,  like  impedance 
control,  or  allows  the  user  to  concentrate  on  non-control  issues,  like 
port  recognition,  by  utilizing  the  control  primitives  originally  sup¬ 
plied  with  the  VAL-II  control  system. 

2.2  Vision  system 

The  recognition  portion  of  this  project  is  accomplished  through  the 
use  of  a  video  camera  and  image  processing  board.  A  Sony  CCD 
video  camera,  Model  XC-38,  provides  the  basis  for  image  acquisition, 
and  was  attached  to  the  third  link  of  the  PUMA-560.  The  camera’s 
small  size  and  low  weight  allows  for  easy  usage  and  minimal  impact 
to  manipulator  dynamics.  An  ITEX  100  Image  Processing  System 
provides  real-time  image  acquisition  and  processing  of  a  digitized 
image.  Each  image  acquired  consists  of  a  pixel  array  size  of  512  X 
480  12-bit  elements.  The  ITEX  board  is  housed  in  a  VAXstation 
III  (CYCLOP)  which  provides  the  computational  power  for  the  vi¬ 
sual  servoing  algorithms  By  acquiring  images  and  analyzing  their 
content,  recognition  and  position  information  regarding  the  refueling 
port  are  derived. 

2.3  Refueling  Apparatus 

The  simulated  refueling  port  is  a  half  scale  mockup  of  a  standard  Uni¬ 
versal  Aerial  Refueling  Receptacle  Slipway  Installation  (UARRSI).  A 
simple  cylinder  mounted  at  the  end  of  a  rod  connected  to  the  force 
sensor  attached  to  the  sixth  link  flange  simulates  the  refueling  nozzle 
These  two  pieces  of  refueling  hardware  were  designed  by  Capt  David 
Duvall  from  the  original  UARRSI  specifications  and  produced  in  the 
AFIT  model  shop  [4j. 

2.4  Force  Sensor 

The  PUMA  wrist  is  fitted  with  a  six  axis  (3  forces,  3  moments)  force 
sensor  to  provide  force  feedback.  The  force  sensor  subsystem,  pro¬ 
duced  by  JR3,  Inc.,  represents  the  state  of  the  art  and  includes  the 
sensor,  a  local  power  supply,  and  processor  electronics  [12].  The  in¬ 
terconnection  between  the  force  sensor  subsystem  and  our  modified 
control  system  (ARCADE)  is  shown  in  Figure  3.  The  sensor  subsys¬ 
tem  functions  independently  via  its  own  operating  system.  Sensor 
commands  are  given  through  the  serial  port  on  the  electronics  pack¬ 
age  Force/moment  sample  rates  are  user  selectable  in  1  ms  incre¬ 
ments.  Obtaining  the  force/moment  vector  requires;  collecting  the 
raw  data,  transformation  to  world  coordinates,  and  correct  scaling 
of  the  sensed  forces  and  moments. 

ARCADE  uses  a  parallel  interface  (DRVll)  between  the  sensor 
electronics  and  the  data  concentrator  (11/73)  to  collect  the  raw  sen¬ 
sor  data.  Data  is  scaled  to  provide  fullscale  force  readings  of  111.2 
newtons  for  Fx  and  Fy  and  222.4  newtons  for  the  Fz  direction.  In¬ 
terface  moments  M*,Mv,Afz  are  limited  to  8.473  newton-meters. 
Scaled  data  is  sent  to  ROBBIE  where  the  sensed  forces  and  mo¬ 
ments  from  the  tool  tip  are  transformed  into  world  coordinates.  The 
transformation  process,  developed  by  Duvall  [4],  includes  correction 
for  gravity  based  on  manipulator  position  and  the  masses  of  the  test 
fixtures. 


Force  sensor  resolution  tests  indicated  that  reposition  inaccura¬ 
cies  caused  repeatability  errors  for  force  measurements.  The  low 
repeatability  mandates  conservative  resolution  limits  of  ±6.0  N  for 
forces  and  ±2.0  N-m  for  moments  [4]  and  the  use  of  a  deadband  lim¬ 
iting  function.  Additional  details  about  force  sensor  operations  and 
calibration  is  contained  in  [18]. 

3  Visual  Servoing 

Our  refueling  concept  requires  that  the  system  autonomously  locate 
the  aerial  refueling  port  and  position  the  nozzle  on  the  slipway.  The 
desire  to  refuel  without  irradiating  the  aircraft  with  lasers  or  ultra¬ 
sonics,  or  extensive  modifications  to  existing  aircraft  systems,  lead 
to  an  investigation  of  visual  techniques.  For  autonomous  motion  to 
the  port  the  system  utilizes  the  visual  information  and  robot  kine¬ 
matics  to  provide  the  desired  trajectory  and  actual  location  to  the 
robot  control  system.  Closing  the  control  loop  around  visual  inputs 
is  commonly  referred  to  as  visual  servoing.  The  area  of  visual  robotic 
servoing  has  been  receiving  greater  attention  as  increased  processing 
power  removes  some  of  the  past  limitations  and  the  demand  for  fix¬ 
tureless  applications  grows.  Previous  visual  servoing  research  [6,8,5] 
provided  insights  but  no  direct  solutions  to  our  unique  problems. 

The  objective  of  the  vision  portion  of  the  refueling  demonstra¬ 
tion  was  for  the  refueler  (PUMA-560)  to  visually  scan  for  the  refu¬ 
eling  port,  and  upon  identification,  provide  the  correct  joint  posi¬ 
tions  (based  on  port  cartesian  coordinates)  necessary  to  servo  the 
end-effector  (refueling  nozzle)  onto  a  stationary  slipway.  A  thorough 
investigation  of  port  recognition  techniques  was  beyond  the  scope  of 
this  initial  research  effort.  Our  recognition  objective  was  to  imple¬ 
ment  a  simple  algorithm  that  could  demonstrate  the  visual  servoing 
concept  and  serve  as  a  baseline  for  future  research.  The  Robotic 
Visual  Servoing  System  (RVSS)  has  been  developed  to  meet  those 
objectives  and  was  able  to  successfully  demonstrate  the  concept  of 
visual  servoing  for  robotic  refueling 

3.1  Receptable  Recognition 

The  refueling  aircraft  is  a  cooperative  target  to  the  refueling  sys¬ 
tem.  Therefore  passive  indicators  should  be  employed  to  reduce  the 
complexity  of  the  recognition  task  The  indicators  chosen  for  this  re¬ 
search  were  three  white  rectangular  strips.  White  was  chosen  so  that 
these  indicators  were  the  whitest  objects  in  the  area  of  the  refueling 
port  (compared  to  the  gray  primer  background).  White  was  not  a 
requirement;  its  use  merely  simplified  the  recognition  algorithm.  A 
different  color  choice  just  requires  resetting  the  threshold  levels  in  the 
histogram  algorithm.  Two  of  the  indicators  where  placed  along  the 
side  of  the  slipway  while  the  other  was  placed  along  the  top,  as  shown 
in  Figure  4.  The  indicators  formed  a  three  sided  box  around  the  re¬ 
fueling  port,  thereby  allowing  the  center  of  the  box  to  be  roughly 
the  desired  port  position  of  contact.  Three  indicators  were  deemed 
the  minimal  number  required  to  provide  center  of  mass,  pose  and 
pattern  uniqueness  data. 

The  location  of  the  port  was  not  know  a  priori  and  the  camera 
had  a  limited  field  of  view.  Therefore  a  necessary  component  of 
the  recognition  processes  was  the  ability  to  search  the  robot’s  range 
space.  A  search  pattern  was  developed  that  servoed  the  PUMA-560 
through  a  predetermined  point-to-point  search  path  [20].  At  each 
point,  an  image  was  acquired  of  the  environment  and  passed  to  the 
recognition  algorithms  for  processing.  In  the  event  that  the  port  was 
determined  not  to  be  in  the  current  image,  the  search  would  continue 
by  servoing  the  PUMA-560  to  the  next  search  point  and  again  an 
image  would  be  acquired  and  analyzed 

After  an  image  was  acquired,  a  histogram  was  performed  on  the 
image  and  the  subsequent  data  stored  in  an  array  Starting  at  the 
white  end  of  the  spectrum  (pixel  value  255)  and  regressing,  an  ele¬ 
ment  of  the  histogram  array  was  compared  to  its  two  neighbors  to 
detect  a  peak.  In  short,  was  the  element  in  question  larger  than  its 
two  neighbors?  If  the  number  of  pixels  with  this  peak  value  or  greater 
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was  larger  than  25,  then  this  pixel  value  was  considered  the  white 
pixel  value.  The  value  25  was  chosen  to  eliminate  the  possibility  of 
small  peak  detection;  peaks  too  small  to  be  the  desired  peak  value. 
In  order  to  provide  flexibility  about  this  value,  the  white  pixel  value 
was  decreased  by  10  percent  of  its  value;  thereby,  providing  a  new 
pixel  value  considered  to  be  the  white  limit.  Any  pixel  whose  value 
was  equal  to  or  greater  than  the  white  limit  was  considered  white, 
everything  else  was  considered  black,  and  the  image  thresholded  ap¬ 
propriately. 

Through  extensive  analysis  of  images  containing  the  three  indi¬ 
cators  but  with  the  port’s  distance  from  the  PUMA  varied,  a  range 
area  of  the  number  of  white  pixels  capable  of  representing  the  three 
indicators  was  determined  If  the  number  of  white  pixels  in  the  image 
was  less  than  100  or  greater  them  100,000,  then  the  refueling  port  was 
considered  not  to  be  in  the  image  and  no  further  recognition  process¬ 
ing  would  be  conducted  on  this  image.  If  the  number  of  white  pixels 
was  within  this  range,  then  the  center  of  mass  of  these  white  pixels 
was  found.  By  manipulating  the  Y  center  of  mass  coordinate,  the 
image  was  divided  in  half.  The  top  half  would  naturally  contain  the 
top  indicator,  and  the  bottom  half  contained  the  two  bottom  indica¬ 
tors.  In  addition  the  bottom  half  of  the  image  was  further  divided 
in  half  to  separate  the  two  bottom  objects.  By  scanning  from  top  to 
bottom  and  left  to  right  in  each  of  these  areas,  the  top,  bottom,  left 
and  right  sides  (row  and  column  locations)  were  derived. 

When  objects  had  been  located  in  each  of  the  three  areas,  simple 
geometric  logic  was  used  to  ensure  that  the  objects  found  were  truly 
the  three  indicators.  These  tests  included  height-to- width  and  width- 
to-height  ratios,  as  well  as  locations  of  object  sides  to  the  sides  of  the 
other  objects.  Upon  satisfying  all  the  logic  requirements,  the  port 
was  considered  identified. 

3.2  Servoing 

Based  on  the  width  of  the  top  indicator,  the  range  of  the  port  to  the 
camera  was  derived  [20].  With  that  value,  the  range  from  the  base 
of  the  PUMA  was  derived  and  its  cartesian  location  found  based  on 
the  forward  kinematics  equations  developed  for  the  first  three  links 
of  the  PUMA  by  Fu,  Gonzalez  and  Lee  [7].  With  the  location  and 
range  parameters  of  the  port  known,  the  inverse  kinematics  were 
derived  through  simple  Law  of  Cosines  equations.  The  desired  joint 
position  of  the  port  was  transmitted  to  the  LSI-1 1/73  which  executed 
a  simple  program  based  on  VAL-II  primitives  to  servo  the  arm  to 
that  location  The  RVSS  could  place  the  nozzle  within  10  cm  of  the 
refueling  port.  That  level  of  accuracy  provides  an  acceptable  starting 
point  for  a  compliant  insertion  algorithm. 

4  Compliant  Control 

The  primary  objective  of  this  portion  of  the  refueling  project  was  to 
accomplish  a  simulated  refueling  port  connection  between  a  refueling 
nozzle  and  the  UARRSI  port  using  the  PUMA-560  robot  under  ac¬ 
tive  compliance  control.  While  visual  servoing  is  a  necessary  part  of 
an  autonomous  refueling  system,  it  is  inappropriate  for  fine  motion 
task  control.  A  simple  analogy  is  putting  a  record  on  a  spindle.  You 
need  your  vision  system  to  provide  an  approximate  location  of  the 
hole  relative  to  the  spindle,  but  the  actual  insertion  is  usually  accom¬ 
plished  by  feel  once  the  record  and  shaft  are  in  contact.  Compliant 
control  provides  a  robot  with  the  ability  to  mimic  a  human’s  ability 
to  feel  i.e.  comply  with  the  environment. 

The  requirement  to  comply  with  the  environment,  and  not  nec¬ 
essarily  command  contact  forces,  was  the  basis  for  our  selection  of 
an  impedance  control  approach.  Several  forms  of  impedance  control 
have  been  evaluated  on  experimental  robots  [11,13]  and  end-effectors 
[14].  The  demonstrated  potential  of  the  approach  lead  to  an  effort  to 
evaluate  impedance  control  for  robotic  refueling  applications.  Imple¬ 
mentation  on  our  PUMA  was  started  in  1988  by  Capt  David  Duvall 
[4].  Subsequent  research  provided  the  refinements  necessary  to  suc¬ 
cessfully  demonstrate  the  advantages  of  compliant  refueling  [18] 


A  basic  premise  of  the  impedance  control  method  is  that  once 
in  contact  with  the  environment  the  manipulator  should  assume  the 
characteristics  of  an  impedance  [9,11].  Therefore  the  task  of  the 
controller  is  to  regulate  the  robot’s  output  impedance,  not  its  motion. 
A  commonly  assumed  target  impedance,  referred  to  as  the  desired 
dynamics  by  Hogan  [9],  is  expressed  as: 

Fint  =  K(x0  -  x)  +  B(v0  -  v)  4-  (1) 

at 

The  damping(R),  stiffness(if),  and  inertia(Z)  of  the  environment  (an 
admittance)  determine  the  most  effective  manipulator  impedance  to 
maintain  during  trajectory  tracking. 

4.1  Implementation 

The  form  of  nonlinear  impedance  control  implemented  in  this  study 
[18]  was  first  derived  by  Hogan  [10]  and  modified  for  the  refueling 
problem  by  Duvall  [4]. 

r„c<=  I(q)J-1M-1K[x0-L(q)}  +  S(q) 
+I(q)J-1M-lB[vo-Jq]  +  V(q) 

~{JT  +  I(q)J-lM-']Fint  (2) 

where: 

q  is  the  measured  joint  position 
q  is  the  computed  joint  velocities 
/  is  the  manipulator  inertia  matrix 
J~l  is  the  inverse  Jacobian  matrix 
M~l  is  the  inverse  desired  mass  matrix 
K  is  the  desired  stiffness  matrix 
xq  is  the  commanded  position 
L  is  the  position  from  forward  kinematics 
5  is  the  gravity  compensation  term 
B  is  the  desired  damping  matrix 
u0  is  the  commanded  velocity 

V  is  the  friction  compensation 
JT  is  the  Jacobian  transpose 

F{nt  is  the  interface  force  between  constraint  and  manipulator 

Feedforward  compensation  (S(?),V(?))  is  used  to  linearize  the 
nonlinear  dynamics  while  multiplication  by  the  inertial  matrix  (I(q)) 
decouples  the  dynamics  and  accounts  for  variations  in  robot  inertia. 
The  inverse  Jacobian  and  inverse  kinematics  solve  the  coordinate 
space  problem. 

While  an  impedance  control  law  can  be  written  for  a  full  6  DOF 
robot  we  restricted  our  initial  research  to  2  DOF.  The  shoulder  and 
elbow  joints  of  the  PUMA  provided  a  rigorous  test  of  the  impedance 
concept  without  the  computational  complexity  required  for  higher 
order  implementation.  The  forward  kinematics,  Jacobian,  and  in¬ 
verse  Jacobian  are  all  calculated  from  symbolic  equations  derived 
by  MACSYMA  [18].  Inertial  parameters,  dynamics  equations,  and 
friction  compensation  are  identical  to  those  employed  in  previous 
gross  motion  control  studies  [17,16].  The  M,  K,  and  B  terms  are 
all  diagonal  matrices  and  are  input  as  control  parameters  for  each 
unique  application  Use  of  a  diagonal  mass  matrix  reduces  M~l 
to  simple  division.  Our  values  for  M,  R,  and  K  were  selected  by 
trial  and  error  with  a  sequence  of  gains  used  to  determine  a  set  of 
values  that  would  indeed  minimize  interface  force  and  still  maintain 
contact  [18].  The  impedance  coefficients  (I  J~l  M~l),  (IJ~1M^lB)y 


1147 


(IJ-xM~lK),  (IJ~1M~1BJ)>  and  (JT 1 J~x M~l)  were  computed 
offline  and  stored  for  each  sample  point.  A  balance  between  maxi¬ 
mum  control  law  sample  rate  and  acceptable  force  sensor  reliability 
was  found  at  5.4  ms  for  our  implementation. 

4.2  Test  Conditions 

A  robotic  manipulator  which  carries  the  refueling  nozzle  must: 

•  move  to  the  slipway, 

•  maintain  surface  contact, 

•  apply  minimal  force  to  the  surface, 

•  overcome  jamming  at  the  port,  and 

•  enter  the  refueling  port. 

To  evaluate  the  impedance  controller’s  ability  to  meet  those  require¬ 
ments  the  robot  was  commanded  to  track  a  simple  trajectory.  The 
test  trajectory,  shown  in  Figure  5,  consists  of  two  segments,  a  verti¬ 
cal  drop  and  a  forward  sweep.  This  trajectory  was  designed  to  drop 
onto  the  refueling  port  slipway  with  commanded  position  going  belou 
the  slipway  surface  and  then  moving  forward  into  the  refueling  port. 
This  motion  provided  continuous  contact  with  the  environment  in 
the  constrained  case.  The  trajectory  was  tailored  to  have  the  correct 
orientation  since  two  degrees  of  freedom  will  not  overcome  misalign¬ 
ments  that  are  out  of  the  xz  plane. 

4.3  Results  and  Analysis 

Prior  to  refueling  concept  demonstration  the  impedance  control  law 
went  through  a  series  of  tests  designed  to  validate  algorithm  perfor¬ 
mance  [18].  An  unconstrained  refueling  test,  i.e.  the  test  trajectory 
without  port  contact,  reaffirmed  the  freespace  trajectory  tracking  ca¬ 
pabilities  of  the  impedance  control  law,  demonstrated  stability  and 
proper  force  sensor  operation  The  ability  to  successfully  follow  the 
constrained  refueling  trajectory  demonstrated  the  feasibility  of  our 
concept  for  autonomous  nozzle  insertion  through  impedance  control. 

Position  errors  from  the  nozzle  insertion  demonstration  are  plot¬ 
ted  in  Figure  6.  Several  important  points  in  the  trajectory  are  easily 
seen  from  these  plots.  Three  points  in  time  will  be  identified  for 
reference.  At  approximately  10  seconds,  the  probe  first  contacts  the 
slipway.  Once  contact  begins,  it  is  never  lost.  At  the  15  second 
point  (half  way  through  the  trajectory),  the  trajectory  changes  from 
vertical  motion  to  horizontal  motion.  At  approximately  28  seconds, 
the  refueling  probe  steps  over  the  entry  lip  and  enters  the  refueling 
port.  These  three  transitions  are  quite  distinctive  in  the  plots  of 
cartesian  position  errors  and  interface  forces.  Once  contact  is  made 
the  position  errors  begin  to  build  up  rapidly.  Commanded  motion 
is  in  the  vertical  direction  only  until  the  15  second  point  so  the  ex¬ 
pected  error  is  only  in  the  z  direction.  However,  the  actual  error  is 
shared  almost  equally  between  the  x  and  z  directions.  The  interface 
force  is  all  in  the  z  direction  as  expected.  The  control  law  splits  this 
input  into  two  torques  based  on  the  impedance  coefficients.  These 
torques  counteract  the  continually  increasing  position  torque  (Fig¬ 
ure  7)  developed  from  the  position  error.  The  resultant  command 
torque  seen  in  Figure  8  exhibits  a  characteristic  that  is  relatively  flat 
over  the  entire  test.  This  active  compliance  capability  is  the  reason 
impedance  control  is  ideal  for  application  to  the  refueling  task.  The 
robot  remains  stable  and  the  contact  force  is  minimized. 

As  the  nozzle  moved  down  the  slipway  incline,  position  errors 
decreased  and  interface  forces  lessened.  This  trend  continued  until 
the  nozzle  encountered  the  metal  housing  around  the  port.  The  lip 
at  the  bottom  of  the  slipway  was  an  obstacle  that  was  detected  and 
compensated  for.  Instead  of  jamming,  the  nozzle  slipped  into  the 
port.  Obviously,  motion  was  much  more  restricted  once  a  connec¬ 
tion  was  completed,  and  data  plots  show  more  damped  operation 
once  the  nozzle  is  connected  to  the  port.  It  is  important  to  note 
that  trajectory  tracking  was  never  lost  even  when  position  error  was 


greatest.  This  test  was  very  repeatable,  but  terminal  position  was 
also  somewhat  dependent  on  the  calibration  point.  Even  under  be¬ 
nign  laboratory  conditions  perfect  knowledge  of  port  position  is  not 
feasible. 

5  Discussion 

While  our  evaluations  clearly  demonstrated  the  feasibility  of  our  re¬ 
fueling  concept  they  also  highlighted  several  limitations.  The  port 
recognition  algorithm  was  not  very  robust  to  changes  in  lighting  con¬ 
ditions.  Early  in  the  project  priority  was  given  to  developing  the 
support  structure  for  visual  servoing  and  not  to  port  recognition 
development.  The  existing  computational  environment  was  also  in¬ 
sufficient  to  support  the  complex  algorithms  necessary  for  brightness 
invariance.  However  the  computational  environment  has  been  up¬ 
graded  with  the  addition  of  an  AP30  Vaccelerator  [2]  and  efforts  to 
implement  a  robust  port  recognition  algorithm  based  on  the  bright¬ 
ness  invariance  techniques  developed  by  Lambert  [15]  are  underway. 
A  dynamic  visual  servoing  algorithm  is  also  under  development  to 
remove  the  stationary  port  restriction. 

To  demonstrate  a  more  realistic  scenario  of  the  actual  nozzle  in¬ 
sertion  procedure  the  degrees  of  freedom  under  impedance  control 
must  be  increased  Once  the  control  law  is  extended  to  the  base  joint 
of  the  PUMA  we  will  be  able  to  remove  the  fixturing  requirements  of 
the  initial  demonstration  and  autonomously  insert  the  nozzle  from 
any  location  in  the  refueler  workspace  With  the  force  sensor  sub¬ 
system  now  fully  integrated  into  ARCADE  the  investigation  of  alter¬ 
native  compliant  control  techniques  is  underway.  Knowledge  of  the 
tradeoffs  between  employment  of  passive  or  active  compliant  control 
schemes  will  provide  valuable  information  for  actual  system  design. 

6  Conclusion 

Robotic  aircraft  refueling  systems  will  inhabit  the  flightline  of  the 
future.  A  refueling  testbed  has  been  developed  at  the  Air  Force 
Institute  of  Technology  Robotic  Systems  Laboratory  to  support  de¬ 
velopment  and  evaluation  of  the  enabling  technologies  that  will  make 
robotic  refueling  a  reality.  Imtial  research  has  concentrated  on  au¬ 
tomating  the  refueling  task  once  the  aircraft  is  within  the  refueler’s 
range  space.  We  have  successfully  demonstrated  a  concept  for  au¬ 
tonomous  nozzle  insertion  that  utilizes  techniques  of  visual  servoing 
and  impedance  control.  Current  research  is  concentrated  on  improv¬ 
ing  the  robustness  of  the  port  recognition  algorithm  and  increasing 
the  number  of  degrees  of  freedom  under  impedance  control.  Further 
research  will  provide  the  refinements  necessary  for  full  scale  develop¬ 
ment. 
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Figure  2:  Robotic  Refueling  Project  Laboratory  Support  Sys¬ 
tem  Diagram 


Figure  3:  Force  Sensor  Subsystem  Block  Diagram 
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Figure  4:  Placement  of  Indicators 


Figure  5:  Refueling  trajectory  from  positive  Y  viewpoint  into 
the  XZ  plane.  Note  the  track  is  beneath,  the  refueling  port. 
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Figure  6:  Trajectory  tracking.  Top:  X  and  Z  against  time. 
Dotted  traces  are  actual  trajectories,  X  on  lower  half  of  plot, 
Z  on  upper.  Bottom:  X  and  Z  in  a  spatial  representation  of 
the  workspace. 
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Figure  7:  Position  Torques  for  the  refueling  trajectory.  Top: 
joint  2  Bottom:  joint  3 
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Figure  8:  Total  Torques  for  the  refueling  trajectory.  Top: 
joint  2  Bottom:  joint  3 


